Survey Descent: A Multipoint Generalization of Gradient Descent for Nonsmooth Optimization

For strongly convex objectives that are smooth, the classical theory of gradient descent ensures linear convergence relative to number evaluations. An analogous nonsmooth is challenging. Even when objective smooth at every iterate, corresponding local models unstable, and cutting planes invoked by traditional remedies difficult bound, leading guarantees sublinear cumulative We instead propose a multipoint generalization iteration for optimization. While our was designed with general in mind, we motivated “max-of-smooth” model captures subdifferential dimension optimality. prove itself max-of-smooth, experiments suggest more phenomenon.